Shape from Silhouette Consensus

Many applications in computer vision require the 3D reconstruction of a shape from its different views. When the available information in the images is just a binary mask segmenting the object, the problem is called shape from silhouette (SfS). As first proposed by Baumgart [1], the shape is usually computed as the maximum volume consistent with the given set of silhouettes. This is called visual hull, a term coined by Laurentini [2] who also studied its theoretical aspects. In real multi-view applications, silhouette masks are extracted from the different views by background subtraction techniques that segment foreground objects. These silhouettes often contain errors due to noise, calibration inaccuracies or background subtraction errors (false alarms and miss-detections) derived from occlusions, illumination changes, moving background or similar colors in the foreground and background. In those cases, when the shape is computed with the visual hull, as the intersection of the back-projected silhouettes in the 3D space, the resulting shapes are incomplete. We propose a shape-from-silhouette algorithm that is robust to inconsistent and incomplete silhouettes. The recovery of the shape that best fits the available data (silhouettes) is formulated as a continuous energy minimization problem. The energy is based on the error between the silhouettes and the shape plus a regularization term. Thanks to the characterization of the visible surface in each view, we consider the error in the volume space. As a result, we obtain an iterative volume-based algorithm that evolves the initial shape to the shape that is in general agreement with the silhouettes, thus being robust to errors in the silhouettes. This work is a reformulation in the continuous case of our previous work [3] and which is more adapted to be implemented in a multi-processor computer architecture such as a graphics processing unit (GPU). Sección en el CEDYA 2011: Sustituir por la sección del Cedya 2011 en la que se